System and method for evaluating the performance of a vehicle operated by a driving automation system

ABSTRACT

Methods and systems for assessing driving autonomous system (DAS) performance based on the telematics data, vehicle type, and/or driving environment. The methods and/or systems may receive driving data indicative of vehicle performance based on DAS operation of the vehicle during a time period; select a portion of the driving data related to at least one performance metric of the DAS operated vehicle during the time period; receive historical DAS performance data that includes at least one performance metric of a vehicle-type of DAS operated vehicle that includes the vehicle; analyze the selected portion of the driving data during the time period with the historical DAS performance data; calculate a DAS score for the vehicle based on the analysis of the selected portion of the driving data during the time period with the historical DAS performance data of the vehicle type; and adjust DAS operation of the vehicle based on the calculated DAS score for the vehicle.

FIELD OF THE DISCLOSURE

The present disclosure generally relates to assessing the performance ofa vehicle operated in a specific environment or operational designdomain (ODD) and specific dynamic driving task (DDT) by a specificdriving automation system (DAS) feature and, more particularly, tosystems and methods for utilizing driving data from vehicles, ODDs,DDTs, environment (e.g., weather, road conditions, traffic), and/orvehicle types to evaluate performance of a vehicle under operation byDASs.

BACKGROUND

A vehicle can be operated by a DAS feature to replace or augmenthuman-operator control commands to drive the vehicle in whole or inpart. Performance characteristics of a vehicle operated by a DAS featurefor specific DDTs such as cruising on a divided highway, may involvemaneuvering and/or handling characteristics or aspects, including, forexample, an acceleration characteristic (e.g., “0-60 mph” measurement),a braking characteristic (e.g. vehicle stoppage at a distance of 45 feetfrom 30 mile per hour), a fuel or battery efficiency characteristic(e.g., 25 miles per gallon or 300 kwh/m, which may be dependent on atype of driving condition (e.g., city, highway), a ground pressurecharacteristic, a power-to-weight ratio, a static stabilitycharacteristic (e.g., rollover resistance, cornering characteristic),and/or other metrics. Metrics for a DAS's feature performance may varygreatly among different types of vehicles, e.g., year, make, model, bodystyle as well as specific vehicles as they learn and age. For example, asportier vehicle type may be expected to maneuver or handle differentlythan a sedan, SUV, or minivan and the respective metrics of the DAS forsuch capabilities among the different vehicle types may be reflective oftheir differences in this regard.

The performance metrics of vehicles operated by driving automationsystem (DAS) may be used by one or more entities for one or morepurposes. For example, prospective purchasers may refer to these metricswhen considering which vehicle type (e.g., make and/or model) and whichDASs or features to buy, such as Tesla's Autopilot, Auto lane-change orAutopark; Audi's Traffic Jam Assist; GM/Cadillac's Super Cruise.Automobile manufacturers may use these metrics to price and/or marketvehicles and DAS features, vehicle insurance providers may use thesemetrics to rate vehicles, and other entities may use the performancemetrics for other purposes. Unfortunately, proclaimed performance ofvehicles operated by DASs may be difficult to assess with respect tomaneuvering and/or handling characteristics, vehicle type, and/ordriving environment. Accordingly, it may be beneficial to provide anevaluation of the performance of vehicles operated by DAS features basedon telematics data that is analyzed according to the make and model ofthe vehicle and/or the ODD or driving context (e.g., drivingenvironment), with respect to purported performance claims and/or to astandard of performance for the vehicles operated by DASs or specificfeatures.

SUMMARY

In accordance with the described embodiments, the disclosure herein isdirected to systems and methods for evaluating a vehicles operated by adriving automation system (DAS).

In one embodiment, a computer-implemented method includes receiving, bya mobile computing device including one or more telematics sensors andoperatively coupled to a vehicle, driving data indicative of vehicleperformance based on driving automation system (DAS) operation of thevehicle during a time period; selecting, by the mobile computing device,a portion of the driving data related to at least one performance metricof the DAS operation of the vehicle during the time period; receiving,by the mobile computing device, historical DAS performance data thatincludes the at least one performance metric of a vehicle type thatincludes the DAS operated vehicle; analyzing, by the mobile computingdevice, the selected portion of the driving data during the time periodwith the historical DAS performance data; calculating, by the mobilecomputing device, a DAS score for the vehicle based on the analysis ofthe selected portion of the driving data during the time period with thehistorical DAS performance data of the vehicle type; and adjusting, bythe mobile computing device, the DAS operation of the vehicle based onthe calculated DAS score for the vehicle.

In another embodiment, a computer-implemented method includes receiving,by a mobile computing device including one or more telematics sensorsand operatively coupled to a vehicle, driving data indicative of vehicleperformance based on driving automation system (DAS) operation of thevehicle during a time period; selecting, by the mobile computing device,a portion of the driving data related to at least one performance metricof the DAS operation of the vehicle during the time period; receiving,by the mobile computing device, historical DAS performance data thatincludes the at least one performance metric of a vehicle type thatincludes the DAS operated vehicle; analyzing, by the mobile computingdevice, the selected portion of the driving data during the time periodwith the received historical DAS performance data; receiving, by themobile computing device, driving context data indicative of a drivingenvironment for the DAS operated vehicle during the time period of theselected portion of the driving data; selecting, by the mobile computingdevice, a portion of the driving context data contemporaneous with theselected portion of the driving data; analyzing, by the mobile computingdevice, the selected portion of the driving data with the selectedportion of the driving context data; calculating, by the mobilecomputing device, a DAS score for the vehicle based on the analysis ofthe selected portion of the driving data during the time period with thehistorical DAS performance data of the vehicle type, and the analysis ofthe selected portion of the driving data with the selected portion ofthe driving context data; and adjusting, by the mobile computing device,the DAS operation of the vehicle based on the calculated DAS score forthe vehicle.

In a further embodiment, a mobile computing device for operativelycoupling to a vehicle operated by a driving automation system (DAS) toevaluate DAS vehicle performance includes one or more processors coupledto one or more memory devices; one or more telematics sensors coupled tothe one or more processors; a user interface coupled to the one or moreprocessors; a communication module operatively coupled to the one ormore processors and facilitating wired and/or wireless communicationwith the mobile computing device; and a scoring module includinginstructions, which when executed by the one or more processors, causesthe system to: receive, via the one or more telematics sensors, drivingdata indicative of vehicle performance based on DAS operation of thevehicle during a time period; select a portion of the driving datarelated to at least one performance metric of the DAS operation of thevehicle during the time period; receive historical DAS performance datathat includes the at least one performance metric of a vehicle type thatincludes the DAS operated vehicle; analyze the selected portion of thedriving data during the time period with the historical DAS performancedata; calculate a DAS score for the vehicle based on the analysis of theselected portion of the driving data during the time period with thehistorical DAS performance data of the vehicle type; and adjust the DASoperation of the vehicle based on the calculated DAS score for thevehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

Advantages will become more apparent to those skilled in the art fromthe following description of the preferred embodiments that have beenshown and described by way of illustration. As will be realized, thepresent embodiments may be capable of other and different embodiments,and their details are capable of modification in various respects.Accordingly, the drawings and description are to be regarded asillustrative in nature and not as restrictive.

The figures described below depict various aspects of the system andmethods disclosed herein. It should be understood that each figuredepicts an embodiment of a particular aspect of the disclosed system andmethods, and that each of the figures is intended to accord with apossible embodiment thereof. Further, wherever possible, the followingdescription refers to the reference numerals included in the followingfigures, in which features depicted in multiple figures are designatedwith consistent reference numerals.

FIG. 1 illustrates a block diagram of an exemplary system including anetwork, a computer server, a mobile computing device, and an on-boardcomputer for monitoring and evaluating DAS operation in accordance withthe described embodiments;

FIG. 2 illustrates a block diagram of an exemplary mobile computingdevice and/or on-board computer;

FIG. 3 illustrates a flow diagram of an exemplary monitoring methodduring DAS operation in accordance with the presently describedembodiments.

FIG. 4 depicts a flow diagram associated with collecting telematics dataand assessing DAS operator performance in accordance with the presentlydescribed embodiments.

FIGS. 5A, 5B, and 5C depict example charts depicting example thresholddata for a variety of vehicles.

FIGS. 6A and 6B depict example interfaces associated with communicatingvehicle operator performance data in accordance with the presentlydescribed embodiments.

DETAILED DESCRIPTION

The methods and systems described herein generally relate to assessingthe performance of a DAS-operated vehicle. DAS operation of the vehiclemay include full control of the vehicle under certain conditions, thatis, complete autonomous operation, or the DAS may assist a humanoperator in operating the vehicle, that is, partial or semi-autonomousoperation. Full autonomous operation may include systems within thevehicle that pilot the vehicle to a destination with or without a humanoperator present, for example, a driverless vehicle. Partial orsemi-autonomous operation may assist the human operator in limited ways,for example, automatic braking or collision avoidance.

More specifically, the methods and systems are directed to evaluating aDAS operator or driver, for example, a DAS driving package or module.Monitored performance of the DAS may be assessed in relation to theproclaimed performance capability of the DAS or a standard ofperformance for the DAS. To score or rank the operation of the DAS, theactual performance of the DAS may be compared to its proclaimedcapabilities or characteristics, and/or to similar or different types ofother DASs or groups thereof. Consideration of the vehicle type that theDAS is integrated therein/therewith, and/or the driving environment(road condition, weather, traffic, etc.) during DAS operation may alsobe included in the evaluation of the performance of the DAS.

Performance features of a DAS are generally related to drivability,where configurations and settings for DAS operation affect the handlingand maneuverability of the vehicle. Some performance characteristics orfeatures may vary with respect to vehicle type, i.e., make and/or model.The performance characteristics may also be affected by and/or adaptedto a driving environment or context during DAS operation, such as thedriving surface and/or its condition, weather, city/rural/highwaydriving, traffic congestion, etc. Some features or aspects of the DASmay be enabled or disabled individually or in groups. For example, amode of operation for the DAS may be selected or adjusted for one ormore DAS features, vehicle types, and/or driving environment.

An analysis of how a DAS facilitates avoiding accidents and/or mitigatesthe severity of accidents may be used to build a database and/or modelof risk assessment. After which, scoring and/or ranking DASs may becompiled and/or updated based upon autonomous or semi-autonomousfunctionality, vehicle type, and/or vehicle usage context, e.g., drivingconditions, road conditions, weather conditions, etc. In one aspect, anevaluation may be performed on how DAS operation compares across vehicletypes, driving context, or proclaimed DAS feature performance statedwithin promotional material. Additional aspects may also facilitate riskassessment and/or premium determination for vehicle insurance policiescovering vehicles with DAS features. For instance, a consumer'sinsurance policy may be based wholly or partially on DAS driving datarelated to a particular vehicle type provided to a vehicle insuranceprovider.

The types of DAS functionality or technology that may be used with thepresent embodiments to replace human operator/driver actions may includeand/or be related to the following types of functionality: (a) fullyautonomous (driverless); (b) limited driver control; (c)vehicle-to-vehicle (V2V) wireless communication; (d)vehicle-to-infrastructure (and/or vice versa) wireless communication;(e) automatic or semi-automatic steering; (f) automatic orsemi-automatic acceleration; (g) automatic or semi-automatic braking;(h) automatic or semi-automatic blind spot monitoring; (i) automatic orsemi-automatic collision warning; (j) adaptive cruise control; (k)automatic or semi-automatic parking/parking assistance; (l) automatic orsemi-automatic collision preparation (windows roll up, seat adjustsupright, brakes pre-charge, etc.); (m) driver acuity/alertnessmonitoring; (n) pedestrian detection; (o) autonomous or semi-autonomousbackup systems; (p) road mapping systems; (q) software security andanti-hacking measures; (r) theft prevention/automatic return; (s)automatic or semi-automatic driving without occupants; and/or otherfunctionality. Additionally or alternatively, the autonomous orsemi-autonomous functionality or technology may include and/or may berelated to: (t) driver alertness or responsive monitoring; (u)pedestrian detection; (v) artificial intelligence and/or back-upsystems; (w) navigation or GPS-related systems; (x) security and/oranti-hacking measures; and/or (y) theft prevention systems.

An evaluation of DAS performance may consider the impact of theautonomous functionality or technology on the likelihood of a vehicleaccident or collision occurring. For instance, a processor may analyzehistorical accident information and/or test data involving vehicleshaving autonomous or semi-autonomous DAS functionality. Factors such asdriving environment and context may be analyzed and/or accounted forthat are related to DAS functionality, accident information, or testdata may include (1) point of impact; (2) type of road; (3) time of day;(4) weather conditions; (5) road construction; (6) type/length of trip;(7) vehicle style; (8) level of pedestrian traffic; (9) level of vehiclecongestion; (10) atypical situations (such as manual traffic signaling);(11) availability of internet connection for the vehicle; and/or otherfactors. These types of factors may also be weighted according tohistorical accident information, predicted accidents, vehicle trends,test data, and/or other considerations.

Benefits of one or more autonomous or semi-autonomous DASfunctionalities or capabilities may be determined, weighted, and/orotherwise characterized. For instance, the benefit of certain autonomousor semi-autonomous DAS functionality may differ with respect to the typeof vehicle integrated therewith. Additionally, or alternatively, thebenefit of some DAS functionality may be greater in city or congestedtraffic, as compared to open road or rural driving traffic. Further,certain autonomous or semi-autonomous DAS functionality may be moreeffective below a certain speed, e.g., during city driving or driving incongestion. Other autonomous or semi-autonomous DAS functionality mayoperate more effectively on the highway and away from city traffic, suchas cruise control. Some autonomous or semi-autonomous DAS functionalitymay be impacted by weather, such as rain or snow, and/or time of day(day light versus night). As an example, fully automatic orsemi-automatic lane detection warnings may be impacted by rain, snow,ice, and/or the amount of sunlight (all of which may impact the imagingor visibility of lane markings painted onto a road surface, and/or roadmarkers or street signs).

Evaluations or rankings of DAS functionality may be adjusted based uponthe type of DAS, vehicle type, and/or driving environment, e.g.,weather, traffic, time of day, etc. Such assessments may further beadjusted based upon the extent of use of the DAS features, includingsettings or modes impacting the operation of the DASs. Informationrelated to the vehicle type and/or driving environment during evaluationmay be included in a comparison to proclaimed performance capabilitiesand/or performance standards of similar or different DAS implementation,e.g., autonomous driving packages, and/or vehicle types.

DAS performance information for a particular vehicle may be gatheredover time and/or via remote wireless communication with the vehicle. Forexample, a mobile computing device may be coupled to the vehicle tomonitor in real-time the DAS and/or the use of DAS features while thevehicle is operating. Other types of monitoring may be performedremotely, such as via wireless communication between the vehicle and aremote server, or wireless communication between a vehicle-coupledcomputing device that is configured to gather autonomous orsemi-autonomous functionality usage information and a remote server.

In one embodiment, an electronic device is operatively coupled to theDAS and may be equipped with one or more sensors to record telematicsdata of a vehicle (for example, acceleration data, braking data,cornering data, and/or other data) during DAS operation of the vehicle.The electronic device may be a portable device such as a mobilecomputing device and/or mobile phone, which may be equipped with one ormore sensors to detect various telematics data during DAS operation ofthe vehicle. Additionally, or alternatively, the electronic deviceand/or the one or more sensors may be fixedly or removably attached tothe DAS-operated vehicle. The telematics data or portions thereof may beevaluated in comparison to historical performance data of a similar typeof DAS-operated vehicle that includes the evaluated DAS. A performancescore for the DAS and/or one or more particular features of the DAS maybe calculated to reflect the DAS's actual performance with respect tothe vehicle type, driving context, proclaimed performance of the DAS,and/or the historical performance of other DASs of similar type.

The electronic device may be configured to transmit the calculated scoreto a remote entity. Alternatively, the electronic device may transmitthe telematics data received via the one or more sensors to the remoteentity, wherein the remote entity may calculate the performance score ofthe DAS. The electronic device and/or the remote entity may include adistribution of the various DAS performance metrics for one or moretypes of DASs. In some instances, the performance score can indicatethresholds for a range of parameters for each of the DAS metrics basedon a particular vehicle type and/or driving context. For example, asports car may have a greater threshold related to acceleration,braking, or cornering than an SUV.

Additionally, in some embodiments, the vehicle may transmit and/orreceive communications to or from external sources, such as othervehicles (V2V), infrastructure (e.g., a bridge, traffic light, railroadcrossing, toll both, marker, sign, or other equipment along the side ofa road or highway), pedestrians, databases, or other information sourcesexternal to the vehicle. Such communication may allow the vehicle toobtain information regarding other vehicles, obstacles, road conditions,or environmental conditions that could not be detected by sensorsdisposed within the vehicle. For example, V2V communication may allow avehicle to identify other vehicles approaching an intersection even whenthe direct line between the vehicle and the other vehicles is obscuredby buildings. As another example, the V2V wireless communication from afirst vehicle to a second vehicle (following the first vehicle) mayindicate that the first vehicle is braking, which may include the degreeto which the vehicle is braking. In response, the second vehicle mayautomatically or autonomously brake in advance of detecting thedeceleration of the first vehicle based upon sensor data.

The DAS performance score may be used for various purposes. Forinstance, the calculated performance score of the DAS may provide a moreobjective perspective of the proclaimed performance capabilities whereDAS performance of similar type vehicles can be grouped, scored, orranked based on actual overall performance and/or individual DASfeatures, aspects, or characteristics. Additionally, the DAS may beadjusted based on the DAS performance score as it relates to the vehicletype and/or driving context. That it, DAS functionality of a particularvehicle type and/or usage within a particular driving context (e.g.,weather, traffic, road condition) can be adjusted based on the DASperformance score as it relates to the corresponding vehicle type,and/or driving context. It should be appreciated that other uses andbenefits may be attained from the calculated DAS performance score.

Although the following detailed description includes numerous differentembodiments, it should be understood that the legal scope of theinvention is defined by the words of the claims set forth further below.The detailed description is to be construed as exemplary only and doesnot describe every possible embodiment, as describing every possibleembodiment would be impractical, if not impossible. One could implementnumerous alternate embodiments, using existing and/or yet-to-bedeveloped technology, which would still fall within the scope of theclaims.

FIG. 1 illustrates a block diagram of an exemplary DAS scoring system100 on which the exemplary methods described herein may be implemented.The high-level architecture includes both hardware and softwareapplications as well as various data communication channels forcommunicating data between the various hardware and software components.The system 100 may be configured into front-end components 102 andback-end components 104. The front-end components 102 may obtaininformation relating to a DAS 108 (e.g., an automobile, truck,motorcycle, etc.) and its surrounding operating environment. An on-boardcomputing device 114 may utilize this information to operate the vehicle108 according to a DAS operation or feature or to assist a human vehicleoperator in operating the vehicle 108. To monitor and/or recordperformance of the vehicle 108, the front-end components 102 may includea mobile computing device 110 (e.g., a smart phone, a tablet computer, aspecial purpose computing device, etc.) to determine when the vehicle isin DAS operation and information regarding the vehicle. One or moresensors 120 may be operably coupled to the vehicle 108 and/or the mobilecomputing device 110 and may communicate with the mobile device 110and/or the on-board computer 114. The front-end components 102 mayfurther process the sensor 120 data using the mobile computing device110 and/or the on-board computer 114. For example, the mobile computingdevice 110 may receive data from the front-end components 102 (e.g., oneor more sensors 120) and determine the use and effectiveness of the DASfeatures of the vehicle 108. In some embodiments of the system 100, thefront-end components 102 may communicate with the back-end components104 via a network 130. Either the on-board computer 114 or the mobiledevice 110 may communicate with the back-end components 104 via thenetwork 130 to allow the back-end components 104 to receive and/orrecord information regarding DAS usage. The back-end components 104 mayalso use one or more servers 140 to receive data from the front-endcomponents 102 and determine the use and effectiveness of DAS featuresof the vehicle 108.

Some of the front-end components 102 may be disposed within orcommunicatively connected to the mobile device 110 and/or the on-boardcomputer 114. The mobile device 110 and/or the on-board computer 114 mayinterface and communicate with the one or more sensors 120 (e.g., avehicle occupant sensor, an ignition sensor, an odometer, a systemclock, a speedometer, a tachometer, an accelerometer, a gyroscope, acompass, a geolocation unit, a camera, a distance sensor, etc.), whichsensors 120 may or may not be incorporated within the mobile device 110.The mobile device 110 and/or the on-board computer 114 may alsointerface and communicate with the one or more external sensors 126associated with the driving environment or context of the DAS. Theexternal sensors 126 may detect conditions relating to infrastructure,weather, traffic, etc., encountered by the DAS 108. The front-endcomponents 102 may further include a communication component 122 totransmit information to and receive information from external sources,including other sensors 126, vehicles, infrastructure, or the back-endcomponents 104. In some embodiments, the mobile device 110 maysupplement or perform all of the functions performed by the on-boardcomputer 114 and/or the mobile device 110 described herein by, forexample, sending or receiving information to/from the server 140 via thenetwork 130. In other embodiments, the on-board computer 114 maysupplement or perform all of the functions of the mobile device 110described herein, in which case no mobile device 110 may be present inthe system 100. Either or both of the mobile device 110 or on-boardcomputer 114 may communicate with the network 130 over links 112 and118, respectively. Additionally, the mobile device 110 and on-boardcomputer 114 may communicate with one another directly over a wiredand/or wireless communication link 116.

The mobile device 110 monitoring performance of the DAS may include adedicated vehicle computing device or a general-use personal computingdevice, cellular phone, smart phone, tablet computer, phablet, wearableelectronic, PDA (personal digital assistant), smart device (glasses,watch, band), pager, and the like, configured for wired or wirelessradio frequency (RF) communication. Although only one mobile device 110is illustrated, it should be understood that a plurality of mobiledevices 110 may be used in some embodiments. Similarly, the on-boardcomputing device 114 may include a dedicated or general-use computingdevice capable of performing functions relating to DAS operation. Theon-board computer 114 may be installed by the original manufacturer ofthe vehicle 108 or as an aftermarket modification or addition to thevehicle 108. In some embodiments or under certain conditions, the mobiledevice 110 or on-board computer 114 may function as thin-client devicesthat outsource some portion of the processing to the server 140.

Any of the sensors 120 may be integrated within the mobile device 110and/or the on-board computing device 114, as well as removably orfixedly installed to the vehicle 108 and disposed in variousconfigurations to provide data related to the DAS features. Some of thesensors 120 may include a global positioning system (GPS) unit or othersatellite-based navigation unit, a radar unit, a Light Detection andRanging unit (LIDAR) unit, an ultrasonic sensor, an infrared sensor, acamera, an accelerometer, a tachometer, a speedometer, as well as anyother sensor that may be appropriate for DAS operation of the vehicle.Some of the sensors 120 (e.g., radar, LIDAR, or camera units) mayactively or passively scan the driving environment for driving contextincluding weather conditions and nearby obstacles (e.g., other vehicles,buildings, pedestrians, etc.), lane markings, signs, or signals. Some ofthe sensors 120 (e.g., GPS, accelerometer, or tachometer units) mayprovide data for determining the location or movement of the vehicle108. Some of the sensors 120 may be directed to the interior or occupantcompartment of the vehicle 108, such as cameras, microphones, pressuresensors, thermometers, or similar sensors to monitor the occupantswithin the vehicle 108. Information generated or received by the sensors120 may be communicated to the on-board computer 114 or the mobiledevice 110 for use in operating, monitoring, and/or evaluatingperformance of the DAS.

In some embodiments, the mobile device 110 and/or communicationcomponent 122 may receive information from the external sensors 126and/or sources, such as other vehicles, weather observation/recordingservices (e.g., weather observation, etc.), or infrastructure (e.g.,roads, bridges, traffic related alerts, etc.). The mobile device 110and/or communication component 122 may also send information regardingthe vehicle 108 to external sources via a transmitter and a receiverdesigned to operate according to desired specifications, such as thededicated short-range communication (DSRC) channel, wireless telephony,Wi-Fi, or other existing or later-developed communications protocols.The information received from the external sensors 126 and/or sourcesmay supplement the data received from other sensors 120 to implement theDAS features. For example, the mobile device 110 and/or thecommunication component 122 may receive information that a DASpositioned ahead of the vehicle 108 is encountering icy road conditionsand/or reducing speed, enabling appropriate adjustments to the DASoperation of the vehicle 108.

In further embodiments, the front-end components 102 may include aninfrastructure communication device 124 for monitoring the status of oneor more infrastructure components. The infrastructure communicationdevice 124 may include or be communicatively connected to the one ormore external sensors 126 for detecting information relating to thecondition of the infrastructure component. The external sensors 126 maygenerate data relating to weather conditions, traffic conditions, oroperating status of the infrastructure component. The infrastructurecommunication device 124 may be configured to receive the generatedsensor data and determine a condition of the infrastructure component,such as weather related conditions (e.g., icy bridge surfaces), roadintegrity, construction, traffic, available parking spaces, etc. Theinfrastructure communication device 124 may further be configured tocommunicate information to the vehicle 108 via the mobile device 110and/or the communication component 122. In some embodiments, theinfrastructure communication device 124 may receive information from thevehicle 108, while, in other embodiments, the infrastructurecommunication device 124 may only transmit information to the vehicle108.

In addition to receiving information from the sensors 120 and externalsensors 126, the mobile device 110 and/or the on-board computer 114 maydirectly or indirectly control the operation of the vehicle 108according to various DAS features. The DAS features may be implementedvia software applications or routines executed by the mobile device 110and/or the on-board computer 114 to control the steering, braking, orthrottle of the vehicle 108. To facilitate such control, the mobiledevice 110 and/or on-board computer 114 may be communicatively connectedto the controls or components of the vehicle 108 by various electricalor electromechanical control components (not shown). In embodimentsinvolving full DAS operation, the vehicle 108 may be operable onlythrough such control components (not shown). In other embodiments, thecontrol components may be disposed within or supplement other vehicleoperator control components (not shown), such as steering wheels,accelerator or brake pedals, or ignition switches.

In some configurations, the front-end components 102 may communicatewith the back-end components 104 via the network 130. The network 130may be a proprietary network, a secure public internet, a virtualprivate network or some other type of network, such as dedicated accesslines, plain ordinary telephone lines, satellite links, cellular datanetworks, and combinations thereof. Where the network 130 comprises theinternet, data communications may take place over the network 130 via aninternet communication protocol.

The back-end components 104 may include one or more servers 140. Eachserver 140 may include one or more computer processors adapted andconfigured to execute various software applications and components ofthe system 100, in addition to other software applications. The server140 may further include a database 146 that may be adapted to store datarelated to the operation of the vehicle 108 and its DAS features. Suchdata may include, for example, an autonomous driving module includingone or more autonomous driving modes relating to handling, maneuvering(e.g., sporty, luxury, comfort, economy, etc.), dates and times ofvehicle use, duration of vehicle use, use and settings of DAS features,speed of the vehicle (e.g., RPM or other tachometer readings), lateraland longitudinal acceleration of the vehicle 108, incidents of or nearcollisions of the vehicle 108, communication between the DAS featuresand external sources, driving context and environmental conditionsaccompanying DAS operation (e.g., weather, traffic, road condition,infrastructure, etc.), errors or failures of DAS features, or other datarelating to use of the vehicle 108 and the DAS features, which may beuploaded to the server 140 via the network 130. The server 140 mayaccess data stored in the database 146 when executing various functionsand tasks associated with evaluating performance of the DAS.

Although the system 100 is shown to include one vehicle 108, one mobiledevice 110, one on-board computer 114, and one server 140, it should beunderstood that different numbers of vehicles 108, mobile devices 110,on-board computers 114, and/or servers 140 may be utilized. For example,the system 100 may include a plurality of servers 140 and multiplemobile devices 110 or on-board computers 114, all of which may beinterconnected via the network 130. Furthermore, the database storage orprocessing performed by the one or more servers 140 may be distributedamong a plurality of servers 140 in an arrangement known as “cloudcomputing.” This configuration may provide various advantages, such asenabling near real-time uploads and downloads of information, as well asperiodic uploads and downloads of information, which may in turn supporta thin-client embodiment of the mobile device 110 or on-board computer114 discussed herein.

The server 140 may include a controller 155 operatively connected to thedatabase 146 via a communication link 156. Although not shown, it shouldbe noted that additional databases may be linked to the controller 155in any known manner. The database(s) may be used for informationrelating to the DAS and/or DAS feature(s), as well as vehicle use. Thecontroller 155 may include a program memory 160, a processor 162 (e.g.,a microcontroller or a microprocessor), a random-access memory (RAM)164, and an input/output (I/O) circuit 166—all of which may beinterconnected via an address/data bus 165. It should be appreciatedthat although only one microprocessor 162 is shown, the controller 155may include multiple microprocessors. Similarly, the memory of thecontroller 155 may include multiple RAMs 164 and multiple programmemories 160. Although the I/O circuit 166 is shown as a single block,it should be appreciated that the I/O circuit 166 may include a numberof different types of I/O circuits. The RAM 164 and program memories 160may be implemented as semiconductor memories, magnetically readablememories, or optically readable memories, for example. The controller155 may also be operatively connected to the network 130 via acommunication link 135.

The server 140 may further include a number of software applicationsstored in a program memory 160. The various software applications on theserver 140 may include an DAS monitoring application 141 for receivingvehicle operating information regarding the vehicle 108 and its DASfeatures, an autonomous feature evaluation application 142 fordetermining the effectiveness (e.g., scoring, ranking) of DAS featuresunder various conditions, and a compatibility evaluation application 143for determining the effectiveness of combinations of DAS features. Thevarious software applications may be executed on the same computerprocessor or on different computer processors.

The various software applications may include various software routinesstored in the program memory 160 to implement various software modulesusing the processor 162. Additionally, or alternatively, the softwareapplications or routines may interact with various hardware modules thatmay be installed within or connected to the server 140. Such modules mayimplement part or all of the various exemplary methods discussed hereinor other related embodiments. Such modules may include a vehicle controlmodule for determining and implementing control decisions to operate thevehicle 108, a system status module for determining the operating statusof DAS features, a monitoring module for monitoring the DAS operation ofthe vehicle 108, a scoring module for determining a score or gradeassociated with DAS performance, a ranking module for determining a rankor comparison associated with DAS performance relating to other vehiclesof similar type (e.g., make, model, autonomous operator), an alertmodule for generating and presenting alerts associated the DAS 108, anidentification module for identifying or verifying the identity of theDAS and/or operator, an information module for obtaining informationregarding the autonomous operator (e.g., autonomous driving package,version, module), an update module for updating the DAS operator and/orDAS feature(s) of the vehicle 108, or other modules.

FIG. 2 illustrates a block diagram of an exemplary mobile device 110and/or an exemplary on-board computer 114 consistent with the system 100illustrated in FIG. 1. The mobile device 110 and/or on-board computer114 may include a controller 204, a display unit 202, a GPS unit 206, acommunication unit 220, and one or more sensors 224 (e.g.,accelerometer, tachometer, a speedometer, gyroscope, etc.). In someembodiments, the mobile device 110 and on-board computer 114 may beintegrated into a single device, or either component may perform thefunctions of both. The mobile device 110 or on-board computer 114 mayinterface with the sensors 120 to receive information regarding the DASoperation of the vehicle 108, its vehicle type, and/or its environment,which information may be used by the DAS features to operate the vehicle108.

Similar to the controller 155, the controller 204 may include a programmemory 208, one or more microcontrollers or microprocessors (MP) 210, amemory device 212 (RAM), and an I/O circuit 216, all of which areinterconnected via an address/data bus 214. The program memory 208 mayinclude an operating system 226, a data storage 228, a plurality ofsoftware applications 230, and/or a plurality of software routines 240.The operating system 226 may include one of a plurality of generalpurpose or mobile platforms, such as the Android™, iOS®, or Windows®systems, developed by Google Inc., Apple Inc., and MicrosoftCorporation, respectively. Alternatively, the operating system 226 maybe a custom operating system designed for DAS operation using theon-board computer 114 and/or the mobile device 110. The data storage 228may include data such as DAS profiles and preferences, application datafor the plurality of applications 230, routine data for the plurality ofroutines 240, and other data related to the DAS features. In someembodiments, the controller 204 may also include, or otherwise becommunicatively connected to, other data storage mechanisms such as oneor more hard disk drives, optical storage drives, solid state storagedevices, etc., that may reside within or external to the vehicle 108.

Similar to the controller 155 in FIG. 1, it should be appreciated thatwhile FIG. 2 depicts one microprocessor 210, the controller 204 mayinclude multiple microprocessors 210. Additionally, the memory of thecontroller 204 may include multiple RAMs 212 and multiple programmemories 208. Further, although FIG. 2 depicts the I/O circuit 216 as asingle block, the I/O circuit 216 may include a number of differenttypes of I/O circuits. For example, the controller 204 may implement theRAM 212 and the program memory 208 as semiconductor memory, magneticallyreadable memory, or optically readable memory.

The one or more processors 210 may be adapted and configured to executeany one of the plurality of software applications 230 or any of theplurality of software routines 240 residing in the program memory 204 orelsewhere. One of the plurality of applications 230 may include a DASoperation application 232 that may be implemented as a series ofmachine-readable instructions for performing the various tasksassociated with implementing one or more of the DAS features accordingto the DAS operation process. Another of the plurality of applications230 may include an autonomous communication application 234 that may beimplemented as a series of machine-readable instructions fortransmitting and receiving DAS information to or from external sourcesvia the communication unit 220. Another application of the plurality ofapplications 230 may include an DAS monitoring application 236 that maybe implemented as a series of machine-readable instructions for sendinginformation regarding DAS operation of the vehicle to the server 140 viathe network 130. Another application of the plurality of applications230 may include an autonomous feature evaluation application 238 thatmay be implemented as a series of machine-readable instructions forsending information regarding DAS operation of the vehicle to the server140 via the network 130.

The plurality of software applications 230 may cooperate with any of theplurality of software routines 240 to perform functions relating to DASoperation, monitoring, scoring, or communication. In some embodiments,one of the plurality of software routines 240 may be an identificationroutine 242 that identifies the type of vehicle for DAS operation.Another of the plurality of software routines may be a configurationroutine 244 to configure the operating parameters of a DAS feature.Another of the plurality of software routines 240 may be a sensorcontrol routine 246 to transmit instructions to a sensor 120 and receivedata from the sensor 120. Still another of the plurality of softwareroutines 240 may be an autonomous control routine 248 that performs atype of autonomous control, such as collision avoidance, lane centering,and/or speed control. In some embodiments, the DAS operation application232 may cause a plurality of autonomous control routines 248 todetermine control actions required for DAS operation. Similarly, one ofthe plurality of software routines 240 may be a monitoring andevaluating routine 250 that monitors and scores DAS operation incomparison to proclaimed features of the DAS. Yet another of theplurality of software routines 240 may be an autonomous communicationroutine 252 for receiving and transmitting information between thevehicle 108 and external sources to facilitate the evaluation of the DASfeatures.

Any of the plurality of software routines 240 may be designed to operateindependently of the software applications 230 or in conjunction withthe software applications 230 to implement modules associated with themethods discussed herein using the microprocessor 210 of the controller204. Additionally, or alternatively, the software applications 230 orsoftware routines 234 may interact with various hardware modules thatmay be installed within or connected to the mobile device 110 or theon-board computer 114. Such modules may implement some or all of thevarious exemplary methods discussed herein or other related embodiments.

For instance, such modules may include a DAS control module fordetermining and implementing control decisions to autonomously operatethe vehicle 108, a system status module for determining the operatingstatus of DAS features, a monitoring module for monitoring the DASoperation of the vehicle 108, a remediation module for correctingabnormal operating states of DAS features, an alert module forgenerating and presenting alerts regarding the DAS 108, anidentification module for identifying or verifying the identity or typeof the DAS and/or DAS operator, an information module for obtaininginformation regarding a DAS operator or vehicle, an update module forupdating a DAS feature of the DAS, and/or other modules.

When executing a DAS operation, the controller 204 of the on-boardcomputer 114 and/or mobile device 110 may implement a vehicle controlmodule by the DAS application 232 to communicate with the sensors 120,126 to receive information regarding the DAS, for example, vehicle type,DAS package, and/or environment; and process that information for DASoperation of the vehicle. In some embodiments including external sourcecommunication via the communication component 122 or the communicationunit 220, the controller 204 may further implement a communicationmodule based upon the autonomous communication application 236 toreceive information from external sources, such as other DASs, smartinfrastructure (e.g., electronically communicating roadways, trafficsignals, or parking structures), or other sources of relevantinformation (e.g., weather, traffic, local amenities, emergencysystems/vehicles). Some external sources of information may be connectedto the controller 204 via the network 130, such as the server 140 orinternet-connected third-party databases (not shown). Although the DASoperation application 232 and the autonomous communication application234 are shown as two separate applications, it is to be understood thatthe functions of the DAS features may be combined or separated into anynumber of the software applications 230 or the software routines 234.

In some embodiments, the controller 204 may further implement amonitoring module by the DAS monitoring application 236 to communicatewith the server 140 to provide information regarding DAS operation. Thismay include information regarding settings or configurations of DASfeatures, data from the sensors 126 regarding the vehicle environment,data from the sensors 120 regarding the response of the DAS to itsenvironment, communications sent or received using the communicationcomponent 122 or the communication unit 220, operating status of the DASoperation application 232 and the autonomous communication application234, and/or commands sent from the on-board computer 114 and/or mobiledevice 110 to the control components (not shown) to autonomously operatethe vehicle. The information may be received and stored by the server140 implementing the DAS information monitoring application 141, and theserver 140 may then determine the effectiveness of the DAS under variousconditions by implementing the feature evaluation application 142 andthe compatibility evaluation application 143.

Some example of sensors 120 operatively coupled to the mobile device 110and/or the on-board computer 114 include a GPS unit 206, anaccelerometer, an optical sensor, a speedometer, a tachometer, athrottle position sensor, a gyroscope, a microphone, an image capturingdevice, a braking detector, etc., which may provide information relatingto DAS operation of the vehicle and/or other purposes. In some specificinstances, the sensors 120 may also be used to monitor vehicle lanedeviation, vehicle swerving, vehicle lane centering, vehicleacceleration along a single axis or multiple axes, and vehicle distanceto other objects. Additionally, external sensors 126 and the sensors 120may also be used to detect driving context, for example, proximatedriving environment, e.g., accompanying weather conditions, trafficcongestion, driving surface, etc. It should be appreciated that thesetypes of sensors and measurable metrics and driving context are merelyexamples and that other types of sensors, measurable metrics, anddriving context are envisioned. Furthermore, the communication unit 220may communicate with other DASs, infrastructure, or other externalsources of information to transmit and receive information relating toDAS operation. The communication unit 220 may communicate with theexternal sources via the network 130 or via any suitable wirelesscommunication protocol network, such as wireless telephony (e.g., GSM,CDMA, LTE, etc.), Wi-Fi (802.11 standards), WiMAX, Bluetooth, infraredor radio frequency communication, etc. Additionally, the communicationunit 220 may provide input signals to the controller 204 via the I/Ocircuit 216. The communication unit 220 may also transmit sensor data,device status information, control signals, and/or other output from thecontroller 204 to one or more external sensors within the vehicle 108,mobile devices 110, on-board computers 114, and/or servers 140.

The mobile device 110 and/or the on-board computer 114 may include auser-input device (not shown) for receiving instructions or informationfrom a vehicle occupant, such as settings relating to a DAS feature. Theuser-input device (not shown) may include a “soft” keyboard that ispresented on the display 202, an external hardware keyboardcommunicating via a wired or a wireless connection (e.g., a Bluetoothkeyboard), an external mouse, a microphone, or any other suitableuser-input device. The user-input device (not shown) may also include amicrophone capable of receiving voice-input of a user.

FIG. 3 illustrates a flow diagram depicting an exemplary monitoringmethod 300 during DAS operation, which may be implemented by the DASevaluation system described herein. The method 300 monitors the DASoperation of the vehicle based upon DAS use. Although this exemplaryembodiment may be primarily performed by the mobile computing device110, the method 300 may be also implemented by the on-board computer114, the server 140, or any combination thereof. Upon receiving anindication of vehicle operation at block 302, the mobile computingdevice 110 may determine the configuration and operating status of theDAS features (including the sensors 120 and the communication component122) at block 304. The identity of the autonomous operator and/orvehicle type may be determined and/or verified at block 306, whichidentity may be used to determine or receive an autonomous operatorprofile at block 308. The autonomous operator profile may containinformation regarding the autonomous operator's ability to operate thevehicle and/or past use of DAS features by the DAS operator. Informationfrom the sensors 120, 126 and/or data from external sources received viathe communication component 122 may be used at block 310 to determineenvironmental conditions, e.g., a driving context, in which the vehicle108 is operating. Together, this information determined at blocks304-310 may be used at block 312 to monitor performance of the DASoperation of the vehicle; from which later may be determined a score orrank associated with similar autonomous operators and/or associatedvehicle types.

Further, the method 300 may continue monitoring DAS operation of thevehicle 108 at block 314, and adjustments may be made based on anydetected changes to the autonomous driver/operator and/or the drivingenvironment. If changes are detected and/or driving conditions aredetermined to have changed at block 314, evaluation criteria for the DASoperation may be adjusted accordingly, in which case the blocks 308 and310 may be repeated. When no changes have been made to the settings, themethod 300 may further check for changes to the environmental conditionsand/or operating status of the DAS features at block 316. If changes aredetermined to have occurred at block 316, corresponding historical DASoperation data may be determined accordingly as at block 314. When nochanges have occurred, the method 300 may determine whether vehicleoperations are ongoing or whether operation is complete at block 318.When vehicle operation is ongoing, the method 300 may continue tomonitor vehicle operation at block 312. When vehicle operation iscomplete, information regarding operation of the vehicle may be recordedat block 320, at which point the method 300 may terminate.

More specifically, at block 302, the mobile computing device 110 and/oron-board computer 114 may receive an indication of vehicle operation.This indication may be received from the autonomous operator (eitherdirectly or through the mobile device 110), and/or it may be generatedautomatically. For example, the mobile device 110 and/or the on-boardcomputer 114 may automatically generate an indication of vehicleoperation when the vehicle starts operation (e.g., upon engine ignition,system power-up, movement of the vehicle 108, etc.). Prior to or uponreceiving the indication of vehicle operation, the mobile computingdevice 110 and/or on-board computer 114 may identify the vehicle type,after which a system check may be initiated as well as the recording ofinformation relating to DAS operation of the vehicle 108.

At block 304, the mobile computing device 110 and/or the on-boardcomputer 114 may determine the configuration and operating status of oneor more DAS features of the vehicle 108. This may include determiningthe configuration, settings, and/or operating status of one or morehardware or software modules for controlling part or all of the vehicleoperation, aftermarket components disposed within the vehicle to provideinformation regarding vehicle operation, and/or sensors 120 coupled tothe vehicle. In some embodiments, a software version, model version,and/or other identification of the feature or sensor may be determined.In further embodiments, the DAS feature may be tested to assess properfunctioning, which may be accomplished using a test routine or othermeans. Additionally, the sensors 120 or the communication component 122may be assessed to determine their operating status (e.g., quality ofcommunication connections, signal quality, noise, responsiveness,accuracy, etc.). In some embodiments, test signals may be sent to one ormore of the sensors 120, responses to which may be received and/orassessed by the on-board computer to determine operating status. Infurther embodiments, signals received from a plurality of sensors may becompared to determine whether any of the sensors are malfunctioning.Additionally, signals received from the sensors may be used, in someembodiments, to calibrate the sensors.

At block 306, the mobile computing device 110 and/or the on-boardcomputer 114 may determine the identity of the autonomous operator. Todetermine the identity of the autonomous operator, the mobile computingdevice 110 and/or the on-board computer 114 may receive and processinformation regarding the autonomous operator. In some embodiments, thereceived information may include sensor data from one or more sensors120 configured to monitor the type of vehicle. For example, informationmay be entered into the mobile device and/or on-board computing deviceregarding the autonomous operator and/or vehicle type to determine theidentity of the autonomous operator. In further embodiments, the mobiledevice and/or on-board computing device may receive information from amobile computing device associated with an occupant of the DAS. Forexample, a mobile phone may connect to the mobile computing device 110and/or the on-board computer 114, which may identify the autonomousoperator and/or vehicle type. Additional steps may be taken to verifythe identity of the autonomous operator and/or vehicle type, such ascomparing an autonomous operator module identifier to a list ofidentified autonomous operators.

At block 308, the mobile device 110 and/or on-board computing device 114may determine and/or access options for the DAS operator and/or thevehicle type based upon the identity of the autonomous operator and/orvehicle type determined at block 306. The autonomous operator profilemay include information regarding options for DAS operation of one ormore vehicles, including information regarding past operation of one ormore vehicles by the autonomous operator. This information may furthercontain past autonomous operator selections of settings for one or moreDAS features for the particular type of vehicle beingmonitored/evaluated. In some embodiments, the mobile device 110 and/oron-board computing device 114 may request or access the autonomousoperator profile based upon the determined identity. In otherembodiments, the mobile device 110 and/or on-board computer 114 maygenerate the autonomous operator profile from information associatedwith the vehicle occupant. The autonomous operator profile may includeinformation relating to one or more driving profiles of the autonomousoperator. For example, the autonomous operator profile may includeinformation relating driving patterns or preferences in a variety ofdriving contexts. In some embodiments, the autonomous operator profilemay include information regarding default settings or features commonlyused in similar type vehicles.

At block 310, the mobile device 110 and/or on-board computer 114 maydetermine the driving environment in which the DAS 108 is operating.Such environmental conditions may include weather, traffic, roadconditions, time of day, location of operation, type of road, and/orother information relevant to operation of the vehicle. Theenvironmental conditions may be determined based upon signals receivedfrom the external sensors 126 and/or resources received through thecommunication component 122, and/or from a combination of other sources.The environmental conditions may then be used in evaluating DASperformance based on driving context and/or vehicle type, and furthercalculating a DAS performance score for use in adjusting DAS operationof the vehicle 108.

At block 312, the mobile device 110 and/or on-board computer 114 maymonitor DAS operation of the vehicle 108, including DAS operationfeature control decisions, signals from the sensors 120, 126, andexternal data from the communication component 122. Monitoring DASoperation may include monitoring data received directly from theautonomous operator, the sensors, and/or other components, as well assummary information regarding the condition, movement, and/orsurrounding environment of the vehicle 108. The mobile device 110 and/oron-board computer 114 may cause the operating data to be stored orrecorded, either locally in the data storage 228 and/or via server 140in the program memory 160 and/or the database 146. Monitoring maycontinue until vehicle operation is complete (e.g., the vehicle hasreached its destination and shut down), including during any updates oradjustments.

At block 314, the mobile computing device 110 and/or on-board computer114 may determine whether any changes have been made to the settings orconfiguration of the DAS features. If such changes or adjustments havebeen made, the mobile computing device 110 and/or on-board computer 114may later adjust the evaluation of the performance of the DAS operatoraccordingly. In some embodiments, minor changes below a minimum changethreshold may be ignored when determining whether any changes have beenmade. In other embodiments, the cumulate effect of a plurality of suchminor changes below the minimum change threshold may be considered as achange at block 314 when the cumulative effect of the minor changesreaches and/or exceeds the minimum change threshold. When no changes tothe settings or configuration of the DAS features are determined to havebeen made at block 314, the mobile computing device 110 and/or theon-board computer 114 may further determine whether any changes in theenvironmental conditions and/or operating status of the DAS features orsensors have occurred.

At block 316, the mobile computing device 110 and/or on-board computer114 may determine whether any changes have occurred to the environmentalconditions of the vehicle 108 and/or the operating status of the DASfeatures, sensors 120, or communication component 122. Such changes mayoccur when weather or traffic conditions change, when sensors 120malfunction or become blocked by debris, and/or when the vehicle 108leaves an area where external data is available via the communicationcomponent 122. When such changes occur, the risk levels associated withcontrol of the vehicle 108 by the vehicle operator and the DAS featuresmay likewise change. Therefore, it may be advantageous to later adjustthe evaluation of the DAS features accordingly to account for suchchanges. Similar to the determination at block 314, minor changes belowa minimum change threshold may be ignored at block 316, unless thecumulative effect of the changes reaches or exceeds the minimum changethreshold. When no changes are determined to have occurred at block 316,the method 300 may continue to monitor the operation of the vehicle 108until vehicle operation is determined to have ended.

At block 318, the mobile computing device 110 and/or the on-boardcomputer 114 may determine whether DAS operation is complete. This mayinclude determining whether a command to shut down the vehicle 108 hasbeen received, whether the vehicle 108 has remained idle at adestination for a predetermined period of time, and/or whether thevehicle operator has exited the vehicle 108. Until operation isdetermined to be complete at block 318 (i.e., when the vehicle trip hasconcluded), the mobile computing device 110 and/or the on-board computer114 may continue to monitor vehicle operation at block 312, as discussedabove. When operation is determined to be complete at block 320, themobile computing device 110 and/or the on-board computer 114 may furthercause a record of the operation of the vehicle 108 to be made or stored.Such records may include operating data (in full or summary form) andmay be used for assessing the autonomous performance of the vehicle. Insome embodiments, records of operating data may be generated and storedcontinually during operation, or partial or completed records may betransmitted to the server 140 to be stored in the database 146.

FIG. 4 depicts a flow diagram of an exemplary method 400 for evaluatingperformance of a DAS. The evaluation may be of a particular type ofvehicle and may include consideration of the driving environment toassess an overall autonomous performance or a specific autonomous aspectassociated with one or more autonomous features. The method 400 may beimplemented through the system depicted in FIGS. 1 and/or 2 and includesome input from vehicle sensors, external sensors, and/or externalsources.

Telematics data associated with DAS performance of a particular vehicleis received by the mobile computing device 110 and/or the on-boardcomputer 114 via one or more sensors 120 at block 402. A portion of thetelematics data relates to the performance of at least one DAS operationaspect of the vehicle, such as maneuvering or handling, for example,braking, accelerating, cornering, etc. The mobile computing device 110and/or the on-board computer 114 may identify at least a portion of thetelematics data that is related to at least one performance metric ofthe DAS at block 404. Threshold data corresponding to the at least oneperformance aspect of a vehicle type that includes the DAS may beacquired from a database at block 406. The threshold data may includehistorical data compiled from previous testing of DASs of similarvehicle type, previous autonomous driving performances, and/or variouscalculations or estimations. Based on the autonomous performancemetric(s), threshold values are determined from the threshold data atblock 408. The threshold values for the performance metric may include arange, for example, a low threshold value representing low usage of theperformance feature, a moderate threshold value representing moderateusage of the performance feature, and a high threshold valuerepresenting high usage of the performance feature. The one or moreportions of the telematics data is compared to the threshold value(s) ofthe threshold data at block 410. An autonomous driving score iscalculated based on the evaluation of the portion(s) of the telematicsdata at block 412. For example, the autonomous driving score may becalculated using the comparison of the portion of the telematics data tothe range of threshold data of the corresponding autonomous performancefeature(s). In some cases, the calculated autonomous driving score maybe based on a percentile(s) of the autonomous performance metric(s) inthe portion of the telematics data. The driving score can be displayedon the mobile computing device, onboard computing device, and/ortransmitted to a remote server for display at block 414.

In another embodiment, the system may receive driving environment datafrom external sensors and/or external sources at block 402. The drivingenvironment data, e.g., driving context data, may include data relatedto the driving conditions during DAS operation of the vehicle. Thedriving context data may include weather conditions, traffic conditions,conditions of the associated driving infrastructure, etc. Threshold datacorresponding to the at least one performance aspect of the vehicle typethat includes the DAS may be adjusted based on the driving environmentdata. Additionally, or alternatively, the compiled threshold datacorresponding to the at least one performance aspect of the vehicle typeincludes consideration of the driving context similar to the drivingcontext that corresponds to the driving context associated with thedriving environment data received by the external sensors and/orexternal sources. The driving score may be calculated based on theevaluation of the portion of the telematics data, the DAS type, and thedriving environment data at block 412. The driving score can bedisplayed on the mobile computing device, onboard computing device,and/or transmitted to a remote server for display at block 414. The DASmay be ranked among similar type DASs based on the comparative drivingscores and/or the DAS may be evaluated in comparison to proclaimedautonomous aspects of similar type DASs, standard DAS aspects of similartype vehicles, and/or driving context.

Evaluation of the DAS operation of the vehicle includes assessment ofone or more performance aspects generally involving vehicle handingand/or maneuvering capabilities that may be characteristics for variousvehicle types, e.g., makes and models. For example, evaluation of DASperformance may be analyzed in comparison to historical DAS performancedata including braking metrics, acceleration metrics, cornering metrics,etc., of similar type vehicles (e.g., make, model) and drivingenvironment. Each metric of the DAS performance data may include one ormore thresholds that represent capability or limit levels for thecorresponding metric for the corresponding vehicle type. The thresholdvalues may be determined by testing DASs and measuring relevant dataduring the testing. The threshold values may also be estimated orcalculated based on existing threshold data, vehicle size data, and/orother factors. The threshold values may have any associated measurementunit. For example, for acceleration, braking, and cornering metrics, thedata and corresponding thresholds may be measured in g-force. Thetesting or calculating to determine the threshold values may bedetermined by the user or obtained from an external resource.

Generally, the higher the threshold value, the more the DAS may beconsidered to being pushed to its theoretical limit of the correspondingmetric. The threshold values may have associated labels indicating thelevel. For example, an SUV of a particular make and model may havethreshold values for a braking metric: a “light” braking, a “moderate”braking, and “severe” braking. The threshold values may be representedby numerical value and/or percentiles for the performance limits (e.g.,the “light” threshold may represent by a range of 0-3.5 and/or representthe 0-34th percentile, the “moderate” threshold may represented by arange of 3.6-7.5 and/or represent the 34th-66th percentile, and the“severe” threshold may be represented by a range of 7.6-10 and/orrepresent the 67th-100th percentile).

Different vehicle types, e.g., make and/or model of vehicles, and DASperformance aspects may have different threshold values for a certainperformance metric. For example, a conversion van of a particular makeand model may have the following threshold values for an accelerationmetric: 2.7 for “light” acceleration, 3.2 for “moderate” acceleration,and 3.8 for “severe” acceleration; and a 2-door sedan of a particularmake and model may have the following threshold values for theacceleration metric: 4.5 for “light” acceleration, 5.5 for “moderate”acceleration, and 7.0 for “severe” acceleration. Accordingly, ingeneral, the lighter, more stable, or otherwise more agile the vehicle,the greater the threshold values may be because that vehicle will beable to handle or maneuver better than heavier, less stable, orotherwise less agile vehicles. After the performance data is compiledand the threshold values determined, the valuation of the autonomousperformance/threshold data may be stored for later retrieval andcalculation of the autonomous driving score and/or rank.

FIGS. 5A-5C depict example threshold values for certain autonomousperformance metrics of various makes and models of vehicles. It shouldbe appreciated that the values illustrated in FIGS. 5A-5C are merelyexamples and may not reflect the true performance metrics of theindicated makes and models of vehicles.

FIG. 5A indicates handling and/or maneuvering (H&M) values 542 for thevarious makes and models of vehicles. Generally, the handling and/ormaneuvering of a vehicle is akin to a cornering performance of a vehicleand is equal to the lateral acceleration in g-force at which rolloverbegins in the most simplified rollover analysis of a vehicle representedby a rigid body without suspension movement or tire deflections. Thefurther down the list of handling and/or maneuvering values 342, thebetter cornering ability of the vehicle (i.e., the more corneringg-forces the vehicle will be able to withstand). For example, thehandling and/or maneuvering (H&M) for a 2017 Ford Econoline van is 0.95and the H&M for a 2017 Chevrolet Impala is 1.40. FIG. 5A furtherindicates threshold values for a cornering performance metric, asmeasured in g-force, which may be calculated based on the correspondinghandling and/or maneuvering testing data or historical performance data.For example, for the 2017 Ford Econoline van, the “light” usagethreshold 343 (e.g., bottom 10th percentile) is 0.250, the “moderate”usage threshold 344 (e.g., 50th percentile) is 0.350, and the “severe”usage threshold 345 (e.g., 90th percentile) is 0.450.

FIG. 5B indicates acceleration values 349 (in the form of 0-60 mphtimes) for various makes and models of vehicles. Generally, the lowerthe 0-60 mph time, the better the acceleration capability of thecorresponding vehicle. Accordingly, the further down the list ofacceleration values 349, the higher the 0-60 mph time and the lesser theacceleration performance of the corresponding vehicle. For example, theacceleration value for a 2017 Volkswagen Routan is 16.00 seconds and theacceleration value for a 2017 Chevrolet Corvette ZR1 is 3.30 seconds.FIG. 5B further indicates threshold values for an accelerationperformance metric, as measured in g-force, which may be calculatedbased on the corresponding 0-60 mph time and/or based on testing data.For example, for the 2017 Toyota Camry, the “light” usage threshold 346(e.g., bottom 10th percentile) is 3.0, the “moderate” usage threshold347 (e.g., 50th percentile) is 4.0, and the “severe” usage threshold 348(e.g., 90th percentile) is 5.0.

FIG. 5C indicates braking values 350 (in the form of stopping distancerequired to decelerate from 60 mph to 0 mph) for various makes andmodels of vehicles. Generally, the further down the list of brakingvalues 350, the longer the stopping distance and the lesser the brakingperformance of the corresponding vehicle. For example, the braking valuefor a 2017 Chevrolet Camaro is 115 feet and the braking value for 2017Ford Econoline is 167 feet. FIG. 5C further indicates threshold valuesfor a braking performance metric, as measured in g-force, which may becalculated based on the corresponding stopping distance and/or based ontesting data. For example, for a 2017 Mitsubishi Eclipse, the “light”usage threshold 351 (e.g., bottom 10th percentile) is 3.5, the“moderate” usage threshold 352 (e.g., 50th percentile) is 4.5, and the“severe” usage threshold 353 (e.g., 90th percentile) is 5.5.

The driving score/rank for the DAS operator based on the evaluationdescribed above may consider the estimated percentiles for the variousperformance metrics, whereby calculation of the driving score/rankutilizes one or more mathematical models, calculations, algorithms,weights, or the like. For example, if the telematics data indicates a45th percentile for cornering, a 90th percentile for acceleration, and a27th percentile for braking, then the calculated autonomous drivingscore/rank may result in an overall result in the 54th percentile (e.g.,an “average” of the three percentiles). In embodiments, variousperformance metrics may be the same or weighted differently.Additionally, the driving score/rank can based on various conventions orscales. For example, the mean driving score can be 100, with numbersabove 100 representing better driving performance and numbers below 100representing worse driving performance. It should be appreciated thatother various algorithms, calculations, assigning conventions, and/orthe like are envisioned.

The calculated driving score/rank may reflect the evaluated DASperformance in a variety of ways. For example, if the DAS operator has abetter-than-average driving score, a corresponding DAS insurance policymay be less than an average premium. If, however, the DAS operator has aless-than-average driving score, the corresponding DAS insurance policymay be more than an average premium.

FIGS. 6A and 6B illustrate example interfaces associated with providingevaluated autonomous driving performances. The mobile device and/oron-board computer may be configured to display the interfaces andreceive selections and inputs via the interfaces. A dedicatedapplication that is configured to operate on the mobile device and/orthe on-board computer may display the interfaces. It should beappreciated that the interfaces are merely examples and that alternativeor additional content is envisioned. Further, it should be appreciatedthat alternative devices or machines may display the example interfaces.

FIG. 6A illustrates an interface 602 that notifies example customer“John D.” of an autonomous driving score (as shown: 110) for the makeand model of a particular vehicle. The interface 602 further includesvarious performance metrics (as shown: cornering, braking, andacceleration) that are calculated from telematics data of the vehicle,as well as percentile indications for the performance metrics.Generally, the DAS is either below or above for the performance metrics.As a result, an insurance provider may calculate an above-averagedriving score of 110, assuming that the average driving score is 100.Any resulting applications (e.g., vehicle insurance quoting) may reflectthe DAS driving score.

FIG. 6B illustrates an interface 604 that notifies example customer“John D.” of a DAS driving score (as shown: 70) associated with aparticular make and model of DAS. The interface 604 further includesvarious performance metrics (as shown: cornering, braking, andacceleration) that are calculated from telematics data of the vehicle,as well as percentile indications for the performance metrics.Generally, the DAS is “above” average for all of the performancemetrics. As a result, an insurance provider may calculate abelow-average driving score of 70, assuming that the average drivingscore is 100. Any resulting applications (e.g., vehicle insurancequoting) may reflect DAS driving score.

Throughout this specification, plural instances may implementcomponents, operations, or structures described as a single instance.Although individual operations of one or more methods are illustratedand described as separate operations, one or more of the individualoperations may be performed concurrently, and nothing requires that theoperations be performed in the order illustrated. Structures andfunctionality presented as separate components in example configurationsmay be implemented as a combined structure or component. Similarly,structures and functionality presented as a single component may beimplemented as separate components. These and other variations,modifications, additions, and improvements fall within the scope of thesubject matter herein.

The systems and methods described herein are directed to an improvementto computer functionality, and improve the functioning of conventionalcomputers. Additionally, certain embodiments are described herein asincluding logic or a number of routines, subroutines, applications, orinstructions. These may constitute either software (e.g., code embodiedon a non-transitory, machine-readable medium) or hardware. In hardware,the routines, etc., are tangible units capable of performing certainoperations and may be configured or arranged in a certain manner. Inexample embodiments, one or more computer systems (e.g., a standalone,client or server computer system) or one or more hardware modules of acomputer system (e.g., a processor or a group of processors) may beconfigured by software (e.g., an application or application portion) asa hardware module that operates to perform certain operations asdescribed herein.

In various embodiments, a hardware module may be implementedmechanically or electronically. For example, a hardware module maycomprise dedicated circuitry or logic that is permanently configured(e.g., as a special-purpose processor, such as a field programmable gatearray (FPGA) or an application-specific integrated circuit (ASIC)) toperform certain operations. A hardware module may also compriseprogrammable logic or circuitry (e.g., as encompassed within ageneral-purpose processor or other programmable processor) that istemporarily configured by software to perform certain operations. Itwill be appreciated that the decision to implement a hardware modulemechanically, in dedicated and permanently configured circuitry, or intemporarily configured circuitry (e.g., configured by software) may bedriven by cost and time considerations.

Accordingly, the term “hardware module” should be understood toencompass a tangible entity, be that an entity that is physicallyconstructed, permanently configured (e.g., hardwired), or temporarilyconfigured (e.g., programmed) to operate in a certain manner or toperform certain operations described herein. Considering embodiments inwhich hardware modules are temporarily configured (e.g., programmed),each of the hardware modules need not be configured or instantiated atany one instance in time. For example, where the hardware modulescomprise a general-purpose processor configured using software, thegeneral-purpose processor may be configured as respective differenthardware modules at different times. Software may accordingly configurea processor, for example, to constitute a particular hardware module atone instance of time and to constitute a different hardware module at adifferent instance of time.

Hardware modules can provide information to, and receive informationfrom, other hardware modules. Accordingly, the described hardwaremodules may be regarded as being communicatively coupled. Where multipleof such hardware modules exist contemporaneously, communications may beachieved through signal transmission (e.g., over appropriate circuitsand buses) that connect the hardware modules. In embodiments in whichmultiple hardware modules are configured or instantiated at differenttimes, communications between such hardware modules may be achieved, forexample, through the storage and retrieval of information in memorystructures to which the multiple hardware modules have access. Forexample, one hardware module may perform an operation and store theoutput of that operation in a memory device to which it iscommunicatively coupled. A further hardware module may then, at a latertime, access the memory device to retrieve and process the storedoutput. Hardware modules may also initiate communications with input oroutput devices, and can operate on a resource (e.g., a collection ofinformation).

The various operations of example methods described herein may beperformed, at least partially, by one or more processors that aretemporarily configured (e.g., by software) or permanently configured toperform the relevant operations. Whether temporarily or permanentlyconfigured, such processors may constitute processor-implemented modulesthat operate to perform one or more operations or functions. The modulesreferred to herein may, in some example embodiments, compriseprocessor-implemented modules.

Similarly, the methods or routines described herein may be at leastpartially processor-implemented. For example, at least some of theoperations of a method may be performed by one or more processors orprocessor-implemented hardware modules. The performance of certain ofthe operations may be distributed among the one or more processors, notonly residing within a single machine, but deployed across a number ofmachines. In some example embodiments, the processor or processors maybe located in a single location (e.g., within a home environment, anoffice environment or as a server farm), while in other embodiments theprocessors may be distributed across a number of locations.

The performance of certain of the operations may be distributed amongthe one or more processors, not only residing within a single machine,but deployed across a number of machines. In some example embodiments,the one or more processors or processor-implemented modules may belocated in a single geographic location (e.g., within a homeenvironment, an office environment, or a server farm). In other exampleembodiments, the one or more processors or processor-implemented modulesmay be distributed across a number of geographic locations.

It should also be understood that, unless a term is expressly defined inthis patent using the sentence “As used herein, the term ‘_(——————)’ ishereby defined to mean . . . ” or a similar sentence, there is no intentto limit the meaning of that term, either expressly or by implication,beyond its plain or ordinary meaning, and such term should not beinterpreted to be limited in scope based on any statement made in anysection of this patent (other than the language of the claims). To theextent that any term recited in the claims at the end of this disclosureis referred to in this disclosure in a manner consistent with a singlemeaning, that is done for sake of clarity only so as to not confuse thereader, and it is not intended that such claim term be limited, byimplication or otherwise, to that single meaning. Finally, unless aclaim element is defined by reciting the word “means” and a functionwithout the recital of any structure, it is not intended that the scopeof any claim element be interpreted based on the application of 35U.S.C. § 112, sixth paragraph.

The term “insurance policy,” as used herein, generally refers to acontract between an insurer and an insured. In exchange for paymentsfrom the insured, the insurer pays for damages to the insured which arecaused by covered perils, acts or events as specified by the language ofthe insurance policy. The payments from the insured are generallyreferred to as “premiums,” and typically are paid on behalf of theinsured upon purchase of the insurance policy or over time at periodicintervals. The amount of the damages payment is generally referred to asa “coverage amount” or a “face amount” of the insurance policy. Aninsurance policy may remain (or have a status or state of) “in-force”while premium payments are made during the term or length of coverage ofthe policy as indicated in the policy. An insurance policy may “lapse”(or have a status or state of “lapsed”), for example, when theparameters of the insurance policy have expired, when premium paymentsare not being paid, when a cash value of a policy falls below an amountspecified in the policy (e.g., for variable life or universal lifeinsurance policies), or if the insured or the insurer cancels thepolicy.

The terms “insurer,” “insuring party,” and “insurance provider” are usedinterchangeably herein to generally refer to a party or entity (e.g., abusiness or other organizational entity) that provides insuranceproducts, e.g., by offering and issuing insurance policies. Typically,but not necessarily, an insurance provider may be an insurance company.

Although the embodiments discussed herein relate to vehicle orautomobile insurance policies, it should be appreciated that aninsurance provider may offer or provide one or more different types ofinsurance policies. Other types of insurance policies may include, forexample, homeowners insurance; condominium owner insurance; renter'sinsurance; life insurance (e.g., whole-life, universal, variable, term);health insurance; disability insurance; long-term care insurance;annuities; business insurance (e.g., property, liability, commercialauto, workers compensation, professional and specialty liability, inlandmarine and mobile property, surety and fidelity bonds); boat insurance;insurance for catastrophic events such as flood, fire, volcano damageand the like; motorcycle insurance; farm and ranch insurance; personalarticle insurance; personal liability insurance; personal umbrellainsurance; community organization insurance (e.g., for associations,religious organizations, cooperatives); and other types of insuranceproducts. In embodiments as described herein, the insurance providersprocess claims related to insurance policies that cover one or moreproperties (e.g., homes, automobiles, personal articles), althoughprocessing other insurance policies is also envisioned.

The terms “insured,” “insured party,” “policyholder,” “customer,”“claimant,” and “potential claimant” are used interchangeably herein torefer to a person, party, or entity (e.g., a business or otherorganizational entity) that is covered by the insurance policy, e.g.,whose insured article or entity (e.g., property, life, health, auto,home, business) is covered by the policy. A “guarantor,” as used herein,generally refers to a person, party or entity that is responsible forpayment of the insurance premiums. The guarantor may or may not be thesame party as the insured, such as in situations when a guarantor haspower of attorney for the insured. An “annuitant,” as referred toherein, generally refers to a person, party or entity that is entitledto receive benefits from an annuity insurance product offered by theinsuring party. The annuitant may or may not be the same party as theguarantor.

Typically, a person or customer (or an agent of the person or customer)of an insurance provider fills out an application for an insurancepolicy. In some cases, the data for an application may be automaticallydetermined or already associated with a potential customer. Theapplication may undergo underwriting to assess the eligibility of theparty and/or desired insured article or entity to be covered by theinsurance policy, and, in some cases, to determine any specific terms orconditions that are to be associated with the insurance policy, e.g.,amount of the premium, riders or exclusions, waivers, and the like. Uponapproval by underwriting, acceptance of the applicant to the terms orconditions, and payment of the initial premium, the insurance policy maybe in-force, (i.e., the policyholder is enrolled).

Unless specifically stated otherwise, discussions herein using wordssuch as “processing,” “computing,” “calculating,” “determining,”“presenting,” “displaying,” or the like may refer to actions orprocesses of a machine (e.g., a computer) that manipulates or transformsdata represented as physical (e.g., electronic, magnetic, or optical)quantities within one or more memories (e.g., volatile memory,non-volatile memory, or a combination thereof), registers, or othermachine components that receive, store, transmit, or displayinformation.

As used herein any reference to “one embodiment” or “an embodiment”means that a particular element, feature, structure, or characteristicdescribed in connection with the embodiment is included in at least oneembodiment. The appearances of the phrase “in one embodiment” in variousplaces in the specification are not necessarily all referring to thesame embodiment.

Some embodiments may be described using the expression “coupled” and“connected” along with their derivatives. For example, some embodimentsmay be described using the term “coupled” to indicate that two or moreelements are in direct physical or electrical contact. The term“coupled,” however, may also mean that two or more elements are not indirect contact with each other, but yet still cooperate or interact witheach other. The embodiments are not limited in this context.

As used herein, the terms “comprises,” “comprising,” “includes,”“including,” “has,” “having” or any other variation thereof, areintended to cover a non-exclusive inclusion. For example, a process,method, article, or apparatus that comprises a list of elements is notnecessarily limited to only those elements but may include otherelements not expressly listed or inherent to such process, method,article, or apparatus. Further, unless expressly stated to the contrary,“or” refers to an inclusive or and not to an exclusive or. For example,a condition A or B is satisfied by any one of the following: A is true(or present) and B is false (or not present), A is false (or notpresent) and B is true (or present), and both A and B are true (orpresent).

In addition, use of the “a” or “an” are employed to describe elementsand components of the embodiments herein. This is done merely forconvenience and to give a general sense of the description. Thisdescription, and the claims that follow, should be read to include oneor at least one and the singular also includes the plural unless it isobvious that it is meant otherwise.

This detailed description is to be construed as examples and does notdescribe every possible embodiment, as describing every possibleembodiment would be impractical, if not impossible. One could implementnumerous alternate embodiments, using either current technology ortechnology developed after the filing date of this application.

What is claimed:
 1. A computer-implemented method comprising: receiving,by a mobile computing device operatively coupled to a vehicle, drivingdata indicative of a driving automation system (DAS) operation of thevehicle during a time period; selecting, by the mobile computing device,a portion of the driving data related to at least one performance metricof the DAS operation of the vehicle during the time period; receiving,by the mobile computing device, historical DAS operation data of othervehicles of a vehicle type that includes the vehicle, the historical DASoperation data including the at least one performance metric; analyzing,by the mobile computing device, the selected portion of the driving datawith the historical DAS operation data to calculate a DAS score for thevehicle; and adjusting, by the mobile computing device, the DASoperation of the vehicle based upon the DAS score for the vehicle. 2.The computer-implemented method of claim 1, wherein the at least oneperformance metric includes one or more of: a cornering metric, anacceleration metric, and a braking metric.
 3. The computer-implementedmethod of claim 1, wherein analyzing the selected portion of the drivingdata with the historical DAS operation data includes comparing the atleast one performance metric with a historical DAS performance scale ofthe at least one performance metric.
 4. The computer-implemented methodof claim 3, wherein the historical DAS performance scale includes one ormore of: a cornering scale, an acceleration scale, and a braking scale.5. The computer-implemented method of claim 4, wherein the historicalDAS performance scale includes one or more thresholds.
 6. Thecomputer-implemented method of claim 1, further comprising: receiving,by the mobile computing device, historical DAS rating data of thevehicle type, the historical DAS rating data including a historical DASscore associated with the vehicle type; and analyzing, by the mobilecomputing device, the historical DAS score of the vehicle type with theDAS score of the vehicle to calculate a DAS rating for the vehicle. 7.The computer-implemented method of claim 1, further comprising:receiving, by the mobile computing device, driving context dataindicative of a driving environment for the vehicle during the timeperiod of the selected portion of the driving data; selecting, by themobile computing device, a portion of the driving context datacontemporaneous with the selected portion of the driving data;analyzing, by the mobile computing device, the selected portion of thedriving data with the selected portion of the driving context data tocalculate a driving context score for the vehicle; and adjusting, by themobile computing device, the DAS score for the vehicle based upon thedriving context score for the vehicle.
 8. The computer-implementedmethod of claim 7, wherein the driving context data include any one ormore of: weather condition, visibility condition, traffic condition, androad condition.
 9. A computer-implemented method comprising: receiving,by a mobile computing device operatively coupled to a vehicle, drivingdata indicative of a driving automation system (DAS) operation of thevehicle during a time period; selecting, by the mobile computing device,a portion of the driving data related to at least one performance metricof the DAS operation of the vehicle during the time period; receiving,by the mobile computing device, historical DAS operation data of othervehicles of a vehicle type that includes the vehicle, the historical DASoperation data including the at least one performance metric; analyzing,by the mobile computing device, the selected portion of the driving datawith the historical DAS operation data; receiving, by the mobilecomputing device, driving context data indicative of a drivingenvironment for the vehicle during the time period of the selectedportion of the driving data; selecting, by the mobile computing device,a portion of the driving context data contemporaneous with the selectedportion of the driving data; analyzing, by the mobile computing device,the selected portion of the driving data with the selected portion ofthe driving context data; calculating, by the mobile computing device, aDAS score for the vehicle based upon analyzing of the selected portionof the driving data with the historical DAS operation data, andanalyzing the selected portion of the driving data with the selectedportion of the driving context data; and adjusting, by the mobilecomputing device, the DAS operation of the vehicle based upon the DASscore for the vehicle.
 10. The computer-implemented method of claim 9,wherein the at least one performance metric includes one or more of: acornering metric, an acceleration metric, and a braking metric.
 11. Thecomputer-implemented method of claim 9, wherein analyzing the selectedportion of the driving data with the historical DAS operation dataincludes comparing the at least one performance metric with a historicalDAS performance scale of the at least one performance metric.
 12. Thecomputer-implemented method of claim 11, wherein the historical DASperformance scale includes one or more of the following: a corneringscale, an acceleration scale, and a braking scale.
 13. Thecomputer-implemented method of claim 12, wherein the historical DASperformance scale includes one or more thresholds.
 14. Thecomputer-implemented method of claim 9, further comprising: receiving,by the mobile computing device, historical DAS rating data of thevehicle type, the historical DAS rating data including a historical DASscore associated with the vehicle type; analyzing, by the mobilecomputing device, the historical DAS score of the vehicle type with theDAS score of the vehicle; and calculating, by the one or moreprocessors, a DAS rating for the vehicle based upon analyzing thehistorical DAS score of the vehicle type with the DAS score of thevehicle.
 15. The computer-implemented method of claim 9, wherein thedriving context data include any one or more of: weather condition,visibility condition, traffic condition, and road condition.
 16. Amobile computing device comprising: one or more processors; one or moretelematics sensors; and one or more memories including instructions,that when executed by the one or more processors, cause the mobilecomputing device to: receive, via the one or more telematics sensors,driving data indicative of a DAS operation of the vehicle during a timeperiod; select a portion of the driving data related to at least oneperformance metric of the DAS operation of the vehicle during the timeperiod; receive historical DAS operation data of other vehicles of avehicle type that includes the DAS operated vehicle, the historical DASoperation data including the at least one performance metric; analyzethe selected portion of the driving data with the historical DASoperation data to calculate a DAS score for the vehicle; and adjust theDAS operation of the vehicle based upon the DAS score for the vehicle.17. The mobile computing device of claim 16, wherein the at least oneperformance metric includes one or more of: a cornering metric, anacceleration metric, and a braking metric.
 18. The mobile computingdevice of claim 16, wherein the one or more memories further includeinstructions, that when executed by the one or more processors, causethe mobile computing device to: receive historical DAS rating data ofthe vehicle type, the historical DAS rating data including a historicalDAS score associated with the vehicle type; and analyze the historicalDAS score of the vehicle type with the DAS score of the vehicle tocalculate a DAS rating for the vehicle.
 19. The mobile computing deviceof claim 16, wherein the one or more memories further includeinstructions, that when executed by the one or more processors, causethe mobile computing device to: receive driving context data indicativeof a driving environment for the vehicle during the time period of theselected portion of the driving data; select a portion of the drivingcontext data contemporaneous with the selected portion of the drivingdata; analyze the selected portion of the driving data with the selectedportion of the driving context data to calculate a driving context scorefor the vehicle; and adjust the DAS score for the vehicle based upon thedriving context score for the vehicle.
 20. The mobile computing deviceof claim 19, wherein the driving context data include any one or moreof: weather condition, visibility condition, traffic condition, and roadcondition.